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1. ABSIBACI A 

An intelligent end effector previously 
developed at JPL has been tested in over 21 
hours of experimental teleoperation. The end 
effector provides local control of gripper j 

clamping force and a 6-degtee-o£-f reedom, j 

wrist mounted force torque sensor. Resolved 
forces and torques were displayed to the test j 
subjects, and the effect of this information 
on their performance of simulated satellite ■ 
servicing tasks was assessed. The experi- j 
enced subjects accomplished the tasks with 
lower levels of RMS forces than intermediate 
and naive subjects, but the force levels were 
apparently uncorrelated with the presence or 
absence of the display. This negative find- 
ing wa3 attributed to the lack of a suitable 
control mode in the manipulator control sys- 
tem. 


Future operations in space such as 
satellite servicing, refueling, and con- 
struction will rely extensively on teleopera- 
tions. Teleoperations will also serve as a 
vital backup capability for any currently 
conceivable level of autonomous robotic func- 
tions. To support these applications work is 
focusing on improving the interface between 
operator and manipulator. Important '•ari- 
acles at this control interface are the 
forces and torques exerted by the operator . 

The Applied Robot ics and Teleoperators 
group at JPL has recently developed a new 
; nte i 1 agent end effector for testing with 
human operators m a simulated satellite ser- 
vice fas* . The JPL-.MV Jmart hand (1,2) is 
equipped with a 6-dof force-torque sensing 
wrist and display system, as well as micro- 
processor control of gripping force. The 
system thu3 provides for controlled gripping 
of dedicate oo;ect J, and feedbacx of force- 
torque information to the operator for avoid- 
ing jams and dangerous stresses on manipu- 
lated objects. 

Recently, the system has been mounted to 
a manipulator arm and tested with human 
operators performing simulated satellite ser- 
vicing tasks. Testing consisted of recording 
forces and torques from the wrist and jaw 
mounted sensors while operators performed 
tasks by teleoperation from the remotely 
located control station. Tasks consisted of 
connecting and disconnecting a fluid cou- 
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pling, simulated module changeout, and 
attempting to exert constant forces and 
torques on the environment. Operators per- 
formed the tasks both with and without the 
force torque display. 

Because of the nature of the tasks, a 
lack of complete information on the manipula- 
tor state and design parameters, and the 
variations inherent in man-in-the-loop sys- 
tems, the analysis of the force/torque data 
presented here tends towards the descriptive 
and anecdotal. However, a quantitative look 
at task performance is provided by reduced 
measures such as root-mean-squared (RMS) and 
Sum of Squared (SOS) forces and torques. 


1. SYSTEM DESCRIPTION 


3-1. Smart Hand jiatlwnta 


The JPL-OMV Smart hand is a one degree 
of freedom gripper with intermeshing jaws 
consisting of parallel plates with a V grove 
section. Each jaw is instrumented with its 
own load cell to measure gripping force. The 
jaws are driven by a DC motor via opposing 
lead screws. 


The entire mechanism is mounted 
manipulator wrist through a 6-dof 
gauge load cell by which interaction 
and torques are measured. Three 
processors are mounted in Lilfi Land for 


to the 
strain 
forces 
micro- 
motor 


control, strain gauge data acquisition and 
processing, and communication. Thus, the 
command interface and force torque feedback 
require only a single full duplex AJ-232 link 
between the hand and tne support equipment. 


A support chassis provides power for 
motor and electronics, and another micro- 
computer for high level control and force 
torque display. A control box is provided 
for operating the hand. The control box has 
controls to set the gripper control mode; to 
give force, rate, and position commands; and 
to set operating parameters such as force and 
rate limits. 


1-Z- Smart Hand Control Software dLd aiSC-Us 


Force and torque servoing of the gripper 
motor takes place entirely in the hand itself 
using the motor control micro-processor. 
Commands from the control box are sent to the 



motor controller via the serial link and the 
communication processor. 

Force and torque information is con- 
-inually sent to the support chassis for 
display by qraphics hardware on a OPT. The 
forces and torques are represented is bar 
qraphs in a star conf iqurat ion (tiqure .) 
which suggests a perspective view of t..e^end 
effector coordinate frame. The software^ pro- 
vides *wo adjustments tor the display: bias 

and ' Fmax' . The bias command memorizes t»e 
current levels of forces/torques, and sub- 
r r act 3 'hem from subsequent values before 
display. This allows strain gauge offsets 
and constant, forces such as gravity no. .0 be 
i ~ splayed . Fmax, seta the magnification 
the lisp lay by specifying the force or torque 


level corresponding to a full bar-graph 
display. The default value of imax is lOv ib 
for testing with human operators, we con- 
sistently set Fmax to 20 lb which pet-"ib-ed 
smaller forces and torques to be resolved on 
the display. 


1- MF.THOPS 

i-i. ?estiaq P retags! 

Testing was performed at the Marshall 
Cnace Flight Center, Huntsville, AL, us*ng 
the * smart hand attached to the Prototype 
Flight Manipulator Arm (PFMA) (3). The r-rMA 
is a 6 -do f arm with a maximum reach of about 
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9 feet. The PFMA was designed for 1G and OG 
testing of autonomous and teleoperated robot- 
ics procedures and devices . For ground based 
testing, removable counterweights are pro- 
vided. 

Teleoperation was set up from a control 
room across the hallway. The operator viewed 
the workplace with three television cameras. 
Two remote controlled pan-tilt-zoom closed 
circuit cameras were located just outside the 
task defined work volume, one aiming parallel 
and one perpendicular to the task board. A 
miniature charge coupled device color camera 
was mounted on the wrist of the robot arm. 
The operator viewed the television signals on 
three monitors, two directly in front of and 
below the line of sight in the control con- 
sole, and one on a large projection televi- 
sion set directly in front and slightly 
elevated. Operators controlled the PFMA via 
task space rate commands given by manipula- 
tion of the 6-dof ' joyball' hand controller 
produced by CAE electronics of Canada. 

Subjects first observed the task board 
and experimental setup in person. In addi- 
tion to general familiarization with the 
task, the subjects were instructed in the 
meaning of the force torque display, and 
exerted forces and torques on the end effec- 
tor jaw 3 with their hands while observing the 
resulting display. 


1.2. Tasks and Subjects 

1.2- 1. XaaXa 

The tasks performed were largely deter- 
mined by the task board accompanying the PFMA 
(3) . In order of the amount of data taken, 
the tasks were: 

1.2- l-l- The. Puroiator fluid coupling 

Thi3 consisted of removal of the fluid 
coupling from its storage socket, translating 
to another female fitting, and inserting the 
connector. The connector wa3 removed by 
grasping the handle, rolling the connector 
approximately 100 degrees, and pulling 
straight back. Insertion was the opposite 
sequence . 

The difficulty in this task lay in pre- 
cise alignment of the connector to avoid jam- 
ming. The task was repeated with and without 
the force/torque display. 

1.2. 1-2. The Module Changeout (fiojj) 

This task consisted of removal and 
insertion of a box shaped module mock-up from 
a close fitting chamber. This task was also 
sensitive to jamming. 

i-2-1-2- Generic £azx£ Control Task (Push- 
ing) 

This task consisted of applying constant 
forces and torques of various magnitudes 
against a stationary object 3uch as the task 
board surface (forces) or the handle of one 
of the ta3k board devices in it3 holder 
(torques) . 

Subjects 

Experimental subjects were employees of 


the Marshall Space Flight Center. Their 
experience level varied from administrative 
employees with no technical experience, to 
engineers who worked with and maintained the 
manipulator and task board on a dally basis. 
Intermediate subjects had used the manipula- 
tor in previous experiments. 


£. 1 . instrumentation and Data Recording 

During task performance, strain gauge 
information was sent via serial link to a 
personal computer which collected the data 
and racorded it on floppy discs for later 
analysis at JPL. Data collected were the raw 
strain gauge signals from the wrist and jaw 
mounted sensors. 

£.2. Analysis Software 

Data analysis at JPL consists of com- 
puter processing to validate, calibrate, and 
resolve raw data recordings, graphics 
software for plotting of forces, torques, and 
gripping force as a function of time, and 
data reduction programs which compute rough 
measures of force control performance (RMS 
and Sum of Squares force signals) . 


£. RESULTS 

£.1. Real Time Data Recording During Task 
Performance 

The first step of data analysis produced 
plots of the precise time history of gripper 
opening; gripping force; x,y,z force; and 
roll, pitch, yaw torques sensed at the wrist. 

Over 21 hours of experimental teleopera- 
tions resulted in a tremendous amount of 
data. The first level of analysis then is to 
present selected task recordings graphically 
and observe what we can from them. From 
there we can proceed to reduce the data and 
compute quantitative measures. 

£.2. Puroiator Fluid Coupling Task 

£.2-1. Experienced Subject 

The first data set is for an engineer 
who worked on the manipulator system and had 
considerable operating experience with the 
arm (with another gripper) . The gripper 
opening and force/torque data from this sub- 
ject performing one repetition of the fluid 
coupling' removal - translation - insertion 
task is shown in figure 2abc. 

The Jaw Opening trace shows that data 
recording began just before the operator 
closed the gripper down on the handle 
preparatory to removal. Clamping force 
immediately increases to about 16C0 units 
(full scale of 2000 is 100 lb), but soon 
varies as forces are applied to the handle 
during removal. The force traces show two 
separate regions of activity corresponding to 
removal and insertion of the fluid coupling. 
These sharp boundaries make it easy to seg- 
ment the data record into removal, transla- 
tion, and insertion phases. 

High force levels (forces in the 
neighborhood of 50 lb were common) were fre- 
quently encountered in this and almost all of 
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Figure 2 . 

Calibrated sensor data for gripper 
status and wrist forces and torques 
during fluid coupling task with an 
experienced operator. Shown are jaw 
opening and clamping force (a), X, Y, 
and Z, axis forces (b) , and pitch, 
roll, and yaw torques (c) . (data set: 
ag0.7) Forces in lb; torques in ft-lb. 


the other data sets. Forces and torques 
change rapidly, and the force and torque 
recordings often appear to consist of a 
series of rectangular pulses at a repetition 
rate of about 0.2 - 0.5 Hz. 


Although there is tremendous variation 
among the recorded data, some interpretation 
of the force and torque record is possible 
from knowledge of the task. Performance of 
the task was complicated by difficulty of 
visual alignment, jams, and inaccurate mani- 
pulator control; and operators may have used 
differing strategies in performing the tasks. 
Thus, the features described below do not 
appear in all of the records, and they are 
intermixed with seemingly random activity. 
However, they appear in enough of the record- 
ings to allow the construction of a schematic 
force torque history (figure 3) of the essen- 
tial components of the task. The purolator 
fluid coupling task is of the twist and lock 
type similar to the common BNC electronic 
connector. Thus, in theory at least, the 
task of removing or inserting it requires 
only Z axis force (insertion/ removal) and 
roll torque (lock/unlock) . All other force 
and torque activity then is the result of 
misaligment, the resulting jamming of the 
mechanism, and attempts by the operator to 
eliminate this condition prior to actual task 
related activity. 
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Figure 3. ^ ■ .. 

Schematic force torque history for 
selected components of wrist 
force/torque for the Purolator connec- 
tor fluid coupling task. Only Z axis 
force and roll torque are 3hown because 
they are the minimal set of 
force/torque components required to 
perform the task (see text) . 

During the translation phase, only grav- 
itational forces are exerted on the wrist. 
Although most of the weight of the fluid cou- 
pling was counteracted by a tool balancer, 
the net gravitational force was still nega- 
tive, and this is visible a3 a shift of about 
-5 lb on the Y axis force during translation. 
A smaller bias is visible in the X axis due 
to the fact that the wri3t was rolled during 
translation (causing a gravitational com- 
ponent in the wrist-oriented X direction) and 
to the force of the trailing coiled hose. 
The large positive extent of Z axis force at 
the end of the removal phase shows the 
operator's strategy of applying a bias force 
in the direction of removal and manipulating 
the coupling until it popped out. 

Torque traces can also be interpreted 
with knowledge of the task. For example, the 
roll torque is positive (i.e., in the direc- 
tion required to open the connector) just 
prior to removal (biasing strategy) and nega- 
tive (i.e., in the direction required to 
close the connector) just prior to completion 
of insertion. These directions are con- 
sistent with the function of the fluid cou- 
pling and were observed in most of the exper- 
imental data. 


£. 1 . 1 . T ypical naive subject 

The selected inexperienced subject was 
an administrator with little previous experi- 
ence on the system. The selected data set 
(figure 4abc) is one recorded after 10 prac- 
tice runs. Task completion took more time 
(200 sec vs. about 170 seconds for the 
experienced operator) . 
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Figure 4 . 

Force torque history for fluid coupling 
task with inexperienced operator. Jaw 
opening and clamping force (a) ; X, Y, 
and Z axis forces (b) ; and pitch, roll, 
and yaw torques (c) . (data set: jr0.6) 

£•2-1. QJL.se.ua alon, 

A distinguishing feature of this data 
set is the group of five pulses of negative Z 
axis force prior to insertion of the cou- 
pling. These pulses correspond to the obser- 
vation by the test conductor that the opera- 
tor was having difficulty aligning the con- 
nector with its socket and was repeatedly 
trying new locations and attempting to insert 
the connector. The strategy was thu3 to 
translate parallel to the task board (the X 
and Y directions) , and then to attempt trans- 
lation into the board (the Z, direction) . In 
the case of the illustrated data, this "tap- 
ping" strategy was tried 5 times in about 20 
seconds (sampling rate of 0.25 Hz) before the 
alignment was correct and the locking opera- 
tion was attempted (first deflection of roll 
torque) . 

The extremely high (over 20 ft-lb; value 
of yaw torque visible during the unlocking 
phase was a jam condition which had to be 
corrected before the fluid coupling could be 
removed, and the rollout-detent-lock feature 
can be seen in roll torque just before the 
end of the task. 

£-2- Module Changeout (Boa) Task 

This task consisted of removing a light 
weight rectangular box (about 1 ft x 1 ft x 2 
ft long) from a close fitting enclosure, mov- 
ing it to a new position and orientation, and 
then moving it back and re-inserting it. 
Compared to the fluid coupling, this is an 


easier task in a more compliant environment 
(the box had significant compliance relative 
to the metal connector) . The completion time 
for this task was relatively short. 

£.2.1. Forces aM torques, etc 

This data set (figure 5abc: experienced 
operator) is selected to show the interaction 
between the forces, torques, and gripping 
force, typical of these tasks. In the jaw 
opening trace, we can see that the gripper 
closed down in an irregular fashion over a 
period of about 10 seconds and that gripping 
force increased noisily during that interval. 
By about 15 seconds into the data set, the 
jaws have closed all the way down onto the 
handle (corresponding to about 40% of the 
full jaw opening) , and the removal is accom- 
plished in relatively short order (positive Z 
axis force, t * 15 - 20 seconds) . Insertion 
is accomplished later (negative Z axis force, 
t - 65 sec), with negligible other forces. 

Roll torque rises to about 20 ft-lb as 
the jaws are closing (t- 10-20 seconds) but 
falls to zero when the object is removed. 
Torques are moderate during insertion (t - 60 
- 70 seconds) and fall to zero before the 
jaws are opened. 

2.2.2. Discussion 

The salient feature of this data set, 
the interaction between jaw closing, clamping 
force, and roll torque illustrates the strong 
coupling between axes in manipulating a free 
object in contact with a rigid non-moving 
environment. The coupling between jaw clos- 
ing and roll torque arises from an initial 
misalignment in the roll axis between the 
claw and the box handle. As the jaws close, 
they create a roll torque, and the two points 
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of contact slide along opposite sides of the 
box handle (which is complying to the roll 
torque) . Surface roughness of the handle 
caused noise in the clamping force and to a 
lesser extent roll torque traces. Finally, 
perhaps due to a roll command, or perhaps due 
to a sudden shift in the handle position, 
grip force momentarily drops to zero and the 
jaws quickly move to the fully closed posi- 
tion . 


£.£. Cpnoric force / torque. CQatXCll task 

£.1.1. Reaulla 

The results of the task board experi- 
ments give a picture of force and torque 
activity in performance of tasks that approx- 
imate anticipated space telerobotic tasks. A 
simpler set of experiments was undertaken to 
determine the operator's ability to control 
forces and torques in a single degree of 
freedom when this was the primary object of 
the task. In these experiments, the gripper 
was locked on a fixed handle, and the opera- 
tor was asked to exert a specified force or 
torque level in a single axi3. The specific 
task consisted of asking the operator to 
exert the force or torque for 30 seconds, to 
return to zero for 10 to 20 seconds, and then 
to re-apply the force/torque for 30 seconds. 
This task was studied only Kith the force 
torque display. 


(b) 



Figure 5. 

Force torque history for module 
changeout task (box) with experienced 
operator, (data set: jgl.4) 


This task proved to be extremely diffi- 
cult because the arm was not provided with 
any type of force control. As rate commands 
were given by the operator, the force or 
torque very rapidly increased to its maximum 
value. When it was possible, force control 
was accomplished by commanding a position at 
which arm compliance would set the desired 
force. Where this was not possible, oscilla- 
tion often occurred. 

Selected records from the generic force 
torque task data illustrate these phenomena 
(figure 6). In the first record (rbj2.1) the 
subject attempted to exert a specified Z axis 
force into the surface of the task board (-20 
lb) . The result is an oscillatory force sig- 
nal indicating the instability of the arm and 
or operator in this control task. This 
oscillation is also present in the second 
example (rbj2.2), in which the 'target' was a 
Z axis force of —40 lb. The oscillation 
appears in at least two different modes: a 
low frequency mode at about 0.5 Hz (rbj2.1) 
and high frequency mode at about 1.4 Hz 
(rbj2 .2) . 

£.4.. 2- PI sous a ion 

In the data presented here, the high 
frequency mode appears to arise from the arm 
control system. It is possible that the low 
frequency mode is due to human operator 
induced oscillation. Further experiments, in 
which additional relevant signals such as 
control input are recorded, are necessary to 
determine if this is the case and under which 
conditions operator induced oscillation can 
arise. The main result of this experiment, 
however, is the realization that a sophisti- 
cated display alone cannot be expected to 
significantly improve operator force control 
performance unless the manipulator control 
system supports some type of force control . 
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The two measures, RMS and SOS 
force/torque, are related to each other but 
differ in one significant respect. RMS 
force/torque differs from SOS force/torque 
because the RMS value has been divided by the 
number of samples in the selected time inter- 
val. This will tend to ignore the difference 
between trials taking a lot of time (jams) 
and trials with brief force/torque records 
(normal insertions) . The SOS measure is not 
normalized in this way and thus reveals a 
measure of the total difficulty of the task. 

£.£.1. EMS Fotcea 



RMS force was plotted for 37 repetitions 
of the purolator fluid coupling task by 
several different operators (figure 7a) . The 
results are broken down by experience level 
of the subject, and whether the task was per- 
formed with or without the force/torque 
display. Force levels are apparently 
un^orrelated for the naive subject. 


RMS Fore* nHdWltho* -WpUy 


Figure 6. 

Selected components froir. the generic 
force/torque task. In these records, 
the subject attempted to maintain con- 
stant force levels for periods of 
approximately 30 seconds. Pushing 
task: 20 lb in the Z axis, in the 
direction into the task board (a: data 
set rbj2.1); 40 lb along the negative Z 
axis (b: data set rbj2.2). 

Cross coupling between axes is evident 
from these data. The axis most effected by Z 
axis force in both records i3 pitch torque. 
This arises naturally from the configuration 
of the arm when grasping a task board handle, 
because the wrist mounted actuator, joint 4 
of the PFMA, is coupled to both Z axis force, 
and pitch torque. 



£.5.. Charact 


Qt Task. 


Reduced data gives a single number 
characterizing the particular task perfor- 
mance. Sum of Squared forces and RMS force, 
are two such measures which integrate the 
magnitude of forces and torques. Since 
forces and torques are associated with jams, 
and excessive forces and torques could result 
in damage to equipment, lower values of these 
force / torque measures can be an indicator 
of 'better' task performance. 


The performance measure used (RMS or Sum 
of Squared (SOS) forces/torques) must be 
carefully considered in light of the nature 
of the task. Since the task consists of two 
distinct periods of force/torque activity 
(removal and insertion) , separated by a rela- 
tively quiet translation phase (figures 
2,3,4), it is not appropriate to use these 
measures to characterize the whole run. For 
example, variations in the translation time 
would modify RMS or SOS measures in ways not 
related to the control of forces and torques 
by the operator. For this reason, RMS and 
SOS measures were computed for only the time 
period corresponding to a given sub-task. 

The reduced data reported here were computed 
during the insertion phase of the task. The 
time interval corresponding to the insertion 
phase was determined from the force/torque 
plots . 
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Figure 7a. 

RMS force/torques for the insertion 
phase of the Purolator fluid coupling 
task . 


£.£.2. £ um a! Squared Forces 

SOS force data were plotted for the same 
repetitions of the task (figure 7b) . The 
overall pattern is similar to the RMS data. 

£.£.£. Reduced Data : Discussion. 

The reduced data apparently show that 
the operator's ability to control forces is 
not significantly affected by the 
force/torque display. Although trends might 
prove significant with more data points, they 
would most lixely be very small ones. Possi- 
ble reasons for the absence of effect are 
that te 3 t subjects could not understand the 
display, or that test subjects could inter- 
pret the display, but could not perform 
corrective control actions. 


The test subjects ability to interpret 
the display was not directly tested, but the 
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Figure 7b. 

Sum of Squared f orce/torques for the 

insertion phase of the Purolator fluid 

coupling task. 

display was explained to each subject, they 
were encouraged to ask questions, and they 
exerted forces and torques on the claw with 
their hands while observing the display. On 
the other hand, since the display was not 
transformed into task space coordinates, the 
information became more and more decoupled 
from the control actions as the hand deviated 
from the straight ahead position. In the 
case of this task, these deviations were 
small, but significant with respect to wrist 
roll . 

The inability to control a perceived 
force was a more serious problem inherent in 
the test setup. The manipulator' 3 control 
system supported only rate control of manipu- 
lator position. Because the environment con- 
sisted of metallic objects of high stiffness, 
small position changes could not be effec- 
tively used to control forces. 

However, the lowest overall force levels 
were achieved by the most experienced opera- 
tors. This was probably due to their profi- 
ciency in precisely positioning the manipula- 
tor to avoid jams. The highest force levels 
occur with the naive subjects, reflecting 
their difficulty with jams and alignment (the 
tapping strategy) . For the intermediate sub- 
jects, force levels appear lower, without the 
display. Although this may not be statisti- 
cally significant, it is possible that 
attempting to use the display information was 
distracting to these subjects and resulted in 
poorer task performance. 

£.£. Operators ' Snh jective RespQns.aa aad 
Feedback 

Although not systematically studied, 
comments and impressions of the subjects 
(often spontaneously offered) were recorded. 
The following is a list of some points 
expressed by individual subjects: 


• Experienced operators found the display 
useful for resolving situations in which 
the fluid coupling was jammed. 

• Operators had difficulty controlling 
forces and torques with the 
manipulator's available rate control (of 
position only) mode (see below) . 

• The Force Torque display needs on-screen 
labels or legends for each force torque 
component bar graph. 

• The display should be resolved into the 
same coordinate system as the control. 

e For the fluid coupling insertion task, 

high accuracy alignment information 
should be displayed rather than forces 
and torques. 


2. RFifiOMMFiNDMIQMS 

Experienced operators found that the 
display of force-torque information was use- 
ful in various phases of telerobotic task 
performance. However, task level testing of 
the JPL-OMV Smart Hand has demonstrated lit- 
tle or no reduction in potentially dasnaging 
forces and torques due to the display of 
force/torque information to the operator. 
The probable reason for this is that althoug K 
force-torque information was made available 
to and understood by the operators, they were 
no better able to control forces and torques 
because that control mode was not provided by 
the manipulator control system. 


Experience with a more precisely con- 
trollable manipulator, the RMS simulator at 
Johnson Space Center, has demonstrated that a 
form of force control can be accomplished 
with small position commands (K. Corker, per- 
sonal communication) . The resulting force is 
a function of the stiffness of the environ- 
ment and the manipulator. Improved control 
modes will be required to go beyond this ad 
hoc method if telemanipulation is to be 
extended into the domain of energetic 
interaction . 

The conclusion to be drawn is that 
display devices, sensors, actuators, and con- 
trol modes for teleoperation cannot be 
designed or fully evaluated in isolation. 
For optimum performance, the full teleopera- 
tion control loop, including the human 
operator, must be considered. 
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